1 research outputs found
Extended Multilingual Protest News Detection -- Shared Task 1, CASE 2021 and 2022
We report results of the CASE 2022 Shared Task 1 on Multilingual Protest
Event Detection. This task is a continuation of CASE 2021 that consists of four
subtasks that are i) document classification, ii) sentence classification, iii)
event sentence coreference identification, and iv) event extraction. The CASE
2022 extension consists of expanding the test data with more data in previously
available languages, namely, English, Hindi, Portuguese, and Spanish, and
adding new test data in Mandarin, Turkish, and Urdu for Sub-task 1, document
classification. The training data from CASE 2021 in English, Portuguese and
Spanish were utilized. Therefore, predicting document labels in Hindi,
Mandarin, Turkish, and Urdu occurs in a zero-shot setting. The CASE 2022
workshop accepts reports on systems developed for predicting test data of CASE
2021 as well. We observe that the best systems submitted by CASE 2022
participants achieve between 79.71 and 84.06 F1-macro for new languages in a
zero-shot setting. The winning approaches are mainly ensembling models and
merging data in multiple languages. The best two submissions on CASE 2021 data
outperform submissions from last year for Subtask 1 and Subtask 2 in all
languages. Only the following scenarios were not outperformed by new
submissions on CASE 2021: Subtask 3 Portuguese \& Subtask 4 English.Comment: To appear in CASE 2022 @ EMNLP 202